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ABSTRACT 



This paper provides information about teacher quality 
proposals that are drawing attention from state and local lawmakers. For each 
proposal, the paper uses the following framework to organize evidence: 
definition or explanation of the policy, rationale, trends, evidence, and key 
issues. The paper demonstrates how the framework would organize evidence 
about setting higher passing scores on teacher licensure examinations, noting 
that some people believe that raising the passing score is a means of 
promoting higher teacher quality. The paper justifies testing related to 
subject matter, pedagogy, content pedagogy, basic skills, and general 
knowledge. Courts have not required states to show that students can learn 
more from teachers with higher scores on licensure examinations. However, a 
small group of studies of student achievement have assessed the importance of 
teachers' tested knowledge, and results show that teachers' tested knowledge 
is an important quality indicator. Raising passing scores will affect the 
quality of the teaching force, though the overall effect will probably be 
modest, and this effort will reduce teacher supply. Recent data suggest . that 
disadvantaged students will be either the beneficiaries or those most 
negatively affected if higher passing scores are implemented. (Contains 33 
references.) (SM) 
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Teaching Policy Handbook 



Higher Passing Scores on Teacher Licensure Examinations 

Working Paper, October 11, 1999 
Analyst: Andrew J. Wayne 



Preface 

The Handbook project will provide reliable knowledge about teacher quality 
proposals that are drawing attention from state and local lawmakers. For each 
proposal, the project uses the following framework to organize evidence: 

1. Definition or explanation of the policy 

2. Rationale 

3. Trends 

4. Evidence 

5. Key Issues 

In addition, individuals who can introduce additional evidence to the analyses will 
have the opportunity to do so. The working paper that follows demonstrates how the 
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framework would organize evidence about setting higher passing scores on teacher 
licensure examinations. 

Proposal 

Establish higher passing scores on teacher licensure examinations. 



Definition or explanation of the policy 

Most states require prospective teachers to pass pencil and paper tests as a 
condition for licensure. States can adjust the difficulty of these tests by raising or 
lowering the passing score. Raising the passing score is seen by some as a means to 
promote higher quality teaching. 



Rationale 

Tests can presumably determine whether prospective teachers possess 
knowledge that enables better teaching. Advocates for raising cutoff scores on tests 
taken by teacher candidates assert that this will spreen out applicants lacking 
necessary knowledge and thereby raise teacher effectiveness and better ensure quality 
teaching for all schoolchildren. Justifications for the five types of tested knowledge 
appear below. 

Subject Matter. Teachers use college-level subject matter knowledge for day- 
to-day decisions, such as how to design lessons, as well as immediate decisions about 
how to respond to student questions. Teachers also use subject matter knowledge to 
design curricula with proper emphases and attention to disciplinary concepts essential 
to continued student learning. Simply put, one cannot teach what one does not know. 

Pedagogy. Decades of academic research have produced a body of 
knowledge about how to teach. Teachers who possess pedagogical knowledge can 
apply it in ways that enhance student learning. 

Subject Matter or “Content ” Pedagogy. Much knowledge about how to teach is 
specific to the particular subject taught. Teachers need subject-specific pedagogical 
knowledge to effectively teach a subject. 

Basic Skills. A competent teacher must be able to read college-level material, 
write cogent essays in correct English, and perform at least those mathematical tasks 
required of high school students. These skills are fundamental to the job of teaching. 

General Knowledge. Teachers who possess broad, general knowledge can 
help students learn by connecting content from various disciplines. Furthermore, 
teachers with more general knowledge can better respond to students’ broad ranging 
curiosity and interests. 



Trends 

[Identify states that recently raised or considered raising cutoff scores and related 
developments.] 
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Evidence 

The courts have not required states to show that students learn more from 
teachers with higher scores on licensure examinations. 1 ^ Nevertheless, a small 
handful of sophisticated studies of student achievement have assessed the importance 
of teachers’ tested knowledge. Six studies are longitudinal - involving pre- and post 
testing - and control for students’ socioeconomic background and other likely 
influences on student achievement. 

Collectively, these studies show that teachers’ tested knowledge is an important 
quality indicator. Although the earliest study raised suspicion about the National 
Teachers Examination, subsequent student achievement studies demonstrated positive 
relationships between student learning and other measures of teachers’ tested 
knowledge. i>: 

The National Teachers Examination. Until recently the National Teachers 
Examination (NTE) was the most widely used licensure examination. Specialized 
teachers took a two-hour NTE Area Examination covering both content and teaching 
methods specific to one of about thirty specialties. Other teachers took the NTE 
Common Examinations, which consumed 3 hours and 15 minutes and included the r 
following four component tests: (1) general principles of pedagogy and psychological 
and social foundations of education, (2) written English expression, (3) social studies, 
literature, and fine arts, and (4) science and mathematics. 

The one student achievement study involving teachers’ NTE scores was 
conducted by Anita Summers and Barbara Wolfe (1977, 1975). The authors sampled 
students in Philadelphia schools in 1971. Among the 627 sixth graders in their 
elementary school sample, students learned less when their teachers scored higher 
on the NTE Common Examinations. The junior high and high school samples 
employed teachers’ NTE Area Examination scores and yielded indeterminate results. 

Summers and Wolfe had controlled for other influences extensively, so their 
findings reinforced other evidence about the NTE. In studies of relationships among 
NTE scores, teachers’ supervisor ratings, and observer ratings, correlations that did 
exist were modest, and a few negative relationships arose (Andrews et al., 1980; 

Ayers and Qualls, 1979; Lovelace and Martin, 1984; Quirk et ah, 1973). 

Verbal Skills. Three subsequent student achievement studies employed tests 
of teachers’ verbal skills. All three yielded reasonably consistent, positive findings. 

Two of the verbal skills studies are reanalyses of old data sets which included 
teachers’ verbal facility scores. Eric Hanushek (1992) reexamined data from an early 
1970s welfare reform experiment and found that teacher “word test” scores had 
affected elementary students’ reading score gains - though not their vocabulary gains 
(see also Mumane and Phillips, 1981). 

The second reanalysis employed the Coleman report data — a very large cross- 
sectional data set from the mid-1960s. Ronald Ehrenberg and Dominic Brewer (1995) 
constructed artificial gain scores by subtracting schools’ lower grade-level 
achievement averages from higher grade-level achievement averages. Teachers’ 
scores on a short verbal facility test explained some school-to-school variation in the 
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gain scores. 

The third verbal skills study employed the reading skill component of a basic 
skills test. Ronald Ferguson (1991, 1 998) took advantage of scores available for over 
half of all Texas teachers tested in statewide teacher testing in 1986 (see also Kain and 
Singleton, 1996). 

Ferguson computed mean teacher test scores for every district. Data on 
students were also aggregated to the district level. The author computed the 
difference, for each district, between the mean achievement score of all third graders 
in 1986 and the mean score of all seventh graders in 1990 — the same cohort of 
children if migration is neglected. The analysis revealed that a district was more 
likely to have a higher difference when the district mean for teachers' test scores was 
higher. Ferguson later reinforced these findings by showing that the gains of each 
district’s elementary students differed from the gains of its secondary students, 
depending on the reading proficiency difference between the district's elementary 
teachers and its secondary teachers. 

Two additional well-designed studies appeared recently. The first study took 
advantage of teachers’ responses to a single multiple choice mathematics test item. 
Rowan, Chiang, and Miller (1997) analyzed nationally representative achievement 
data from the National Educational Longitudinal Study of 1988. The teacher survey ! 
included a single, high school level mathematics test item. The researchers found that 
students whose teachers answered the item correctly posted larger mathematics gains 
between eighth and tenth grade. 

A second recent study capitalized on Alabama personnel records, some of 
which contained teachers’ ACT College entrance examination test scores. Ronald 
Ferguson and Helen Ladd (1996) used composite ACT scores. These scores 
combined English, mathematics, social studies reading, and natural sciences reading 
components (American College Testing, 1989). 

Their initial analysis showed that student reading score gains from third grade 
to fourth grade were positively related to the average teacher ACT score at the 
students’ school. The relationship was unclear for math score gains. Their second 
analysis used district level averages from 127 school districts. The authors created 
artificial gain scores using achievement data from third and fourth graders, and eighth 
and ninth graders, all for the 1990-91 school year. The average differences in each 
district between the math scores of younger students and older students were 
positively related to the average teacher ACT score in the district. 

Summary. The evidence shows that tested knowledge is a relevant indicator of 
teacher quality. The research findings do not map cleanly onto currently used 
licensure tests, but Tables 1 and 2 show that basic skills tests and subject matter tests 
are reasonably well supported. 

Table 1 tentatively connects the determinate findings involving elementary 
students to the three types of licensure examinations for elementary teachers. Table 2 
depicts the same tool for secondary students and teachers. 

The connections drawn do involve some leap-of-faith generalizations. The 
reading component of the TECAT, for example, represents only one component of a 
typical basic skills test. Conversely, the connections also leap from general to 
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particular. The Ferguson and Ladd study that used teachers’ ACT composite scores, 
for example, connects to three columns of Table 2. We cannot know whether 
Alabama’s secondary school students learned more because of teachers’ knowledge of 
the subject taught, general knowledge, or basic skills. 
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Key Issues 

Effect on Teacher Quality. Raising passing scores will affect the quality of the 
teaching force, but the overall effect is probably modest. Teachers just above the old 
passing score will presumably be replaced by teachers just above the new passing 
score. In addition, student achievement studies show that the overall contribution of 
teachers to student learning is much larger than teachers’ tested knowledge can 
explain (Goldhaber & Brewer, 1997; Hanushek, 1995; Hedges, et al., 1996; Mumane, 
1975, Mumane and Phillips, 1981; Rivkin, et al., 1998; Sanders and Rivers, 1996). 
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Effect on Teacher Supply. Raising passing scores necessarily reduces supply. 
Some evidence from the Educational Testing Service (ETS) helps gauge that effect. 
Gitomer et al. (1999) created a data set using tens of thousands of licensure test 
records collected from 1994 through 1997. If every state using Praxis I 121 had applied 
the cutoff score from the state with the highest cutoff score, the pool of test passers 
would have shrunk by one-third. The same analysis performed for Praxis IP 1 shows 
that the pool of Praxis II passers would have shrunk by one quarter. The authors also 
described the pool of passers according to its racial makeup. In the Praxis I and 
Praxis II analyses, minority representation in the pool of test passers fell by almost 
one third, and by one fifth, respectively. 

The ETS estimates of the effects of higher passing scores on teacher supply 
may overstate the impact. With higher cutoff scores actually in place, low scoring 
aspirants have strong incentives to acquire the tested knowledge. Nonetheless, 
substantially raising pass scores on teacher tests will have a big effect on the number 
of people licensed to teach. The reduction in the pool of teacher candidates will be 
particularly great among persons who are African American or Latino. 

Effect on Disadvantaged Schoolchildren. In the past, low-income and 
minority students were the most likely to have teachers with low tested knowledge 
(Ehrenberg & Brewer, 1995; Summers and Wolfe, 1976). Those students therefore 
had the most to gain if higher pass scores were to result in the recruitment and 
retention of teachers with teachers who did well on such tests. However, low income 
and minority students have the most to lose from licensure test cutoffs if those 
individuals who then teach them have no preparation to teach or are unlicensed and 
have even lower verbal ability and content knowledge. 

Recent tabulations indicate that disadvantaged schoolchildren will be either the 
beneficiaries or those most negatively affected if higher passing scores are 
implemented. Low-scoring teachers remain concentrated with both minority and low- 
income schoolchildren (Ferguson, 1998; Kain and Singleton, 1996; Rowan et al., 
1997). In addition, matched records from the Educational Testing Service show that 
teachers who pass the Praxis II examination for special education have markedly 
lower SAT scores than teachers who pass other Praxis II examinations (Gitomer et al., 
1999). 

Concluding Observations 

It seems clear that states will continue to increase the passing scores on various tests of teacher 
knowledge. Of the tests now in use, we have evidence that subject matter and verbal skills TESTS 
are related to teacher effectiveness. This does not mean that other tests do not capture aspects of 
effective teaching, only that we cannot say from existing research that this is so. 

While students whose teachers have high verbal skills and know their subject matter are more likely 
to achieve at somewhat higher levels than students whose teachers do less well on relevant tests, 
EXISTING EVIDENCE IS NOT SUFFICIENT to determine what the gain in student achievement 
would be as a result of a given increase in the passing score. 

It is possible that the quality of the teaching corps would actually decline in hard-to-staff schools as 
passing scores are raised above current levels. Districts, confronted with fewer licensed applicants 
may persuade states to issue emergency permits. Such declines could be avoided I passing scores 



http://www.ericsp.org/digests/LicExams.htm 



8 



12/06/2000 



NATIONAL PARTNERSHIP FOR EXCELLENCE 



Page 8 of 1 1 



increases were complemented with salary increases or increases in other incentives to teach. 

The fact that pencil and paper tests of knowledge are weak, if positive in some cases, indicators, has 
led an increasing number of people wanting to raise the quality to urge that prospective candidates be 
required to. Such performance assessment would include tests of knowledge but the overall score of 
a prospective teacher would depend on several measures, including assessments of actual classroom 
teaching performance. 
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^ Defendants in employment discrimination cases must provide evidence that employment criteria are job-related. For 
teacher licensure examinations, documentation from a proper test development process suffices. At the Educational 
Testing Service, for instance, developers analyze job requirements by surveying practicing educators. These analyses 
inform the efforts of committees of experienced teachers and teacher-educators who develop test specifications. Finally, 
committee members and other teachers collaborate with subject matter and test development experts to specify actual 
questions. The courts also require information about how passing scores are set. For more information, see Educational 
Testing Service (1999a, 1999b, 1999c). 

^ Praxis I is a basic skills test developed by the Educational Testing Service and currently used by about twenty states. 
^ Praxis II is a licensure test developed by the Education Testing Service. Praxis II examinations cover content and 
pedagogical knowledge in a specific subject field. 
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